Add unit tests for utils/process_result.py with consolidated pytest CI by Copilot · Pull Request #197 · SemiAnalysisAI/InferenceX

Copilot · 2025-11-06T22:14:41Z

Adds comprehensive test coverage for the benchmark result processing script with automated CI verification. Refactors process_result.py to use testable functions instead of top-level script execution.

Changes

utils/process_result.py: Refactored for better testability
- Extracted process_benchmark_result() function containing core processing logic
- Added main() function for CLI execution
- Maintains backward compatibility with existing CI workflows
- Eliminates security risks from script execution patterns
utils/test_process_result.py: 15 unit tests covering:
- Environment variable parsing (TP, EP_SIZE, PREFILL_GPUS, DECODE_GPUS, etc.)
- Throughput calculations per GPU with disaggregated prefill/decode
- Metric conversions (ms→s, tpot_ms→intvty)
- Conditional field handling (MTP_MODE)
- Output JSON structure validation
- Tests import and call functions directly (no exec())
.github/workflows/test-utils.yml: Consolidated pytest CI workflow
- Runs all utils tests (73 matrix-logic tests + 15 process_result tests = 88 total)
- Triggers on any Python file changes in utils directory
- Python 3.12, skips draft PRs
- Replaces separate test-matrix-logic.yml and test-utils-process-result.yml workflows
.gitignore: Added pytest cache and Python bytecode exclusions

Test Example

def test_throughput_calculations(basic_env_vars):
    """Verifies per-GPU throughput with explicit prefill/decode GPU counts."""
    benchmark_result = {
        'max_concurrency': 10,
        'model_id': 'test/model',
        'total_token_throughput': 24000.0,
        'output_throughput': 8000.0
    }
    
    env_vars = basic_env_vars.copy()
    env_vars['TP'] = '4'
    env_vars['PREFILL_GPUS'] = '2'
    env_vars['DECODE_GPUS'] = '2'
    
    result = process_benchmark_result(benchmark_result, env_vars)
    
    # tput_per_gpu = total / tp_size = 24000/4 = 6000
    assert result['tput_per_gpu'] == 6000.0
    # output_tput_per_gpu = output / decode_gpus = 8000/2 = 4000
    assert result['output_tput_per_gpu'] == 4000.0
    # input_tput_per_gpu = (total - output) / prefill_gpus = 16000/2 = 8000
    assert result['input_tput_per_gpu'] == 8000.0

All 88 tests pass. No security alerts from CodeQL.

Fixes InferenceMAX/InferenceMAX#196

Original prompt

This section details on the original issue you should resolve

<issue_title>write unit test for utils/process_result.py & add pytest CI for it</issue_title>
<issue_description></issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes InferenceMAX/InferenceMAX#196

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot

Pull Request Overview

This PR adds comprehensive unit testing for utils/process_result.py and sets up automated CI testing via GitHub Actions. The implementation includes 15 test cases covering core functionality such as benchmark result processing, metric conversions (milliseconds to seconds, TPOT to interactivity), GPU calculations, and various configuration scenarios. The tests use dynamic path resolution and proper fixtures for better maintainability.

Key changes:

Created test_process_result.py with 15 comprehensive unit tests
Added GitHub Actions workflow for automated pytest execution
Implemented dynamic path resolution for better portability

Reviewed Changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 3 comments.

File	Description
`utils/test_process_result.py`	New test file with 15 unit tests covering all major functionality of process_result.py
`.github/workflows/test-utils-process-result.yml`	GitHub Actions workflow configuration for running pytest on relevant file changes

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-06T22:25:43Z

+
+jobs:
+  test:
+    if: github.event_name != 'pull_request' || github.event.pull_request.draft != true


[nitpick] The condition skips tests on draft PRs, but the PR title contains '[WIP]' suggesting this is a work-in-progress pull request. Consider removing this condition or updating the workflow to run on draft PRs to ensure tests are executed during development.

Suggested change

if: github.event_name != 'pull_request' || github.event.pull_request.draft != true

Copilot · 2025-11-06T22:25:44Z

+    try:
+        with patch.dict(os.environ, basic_env_vars):
+            # Import and execute the script
+            exec(open(PROCESS_RESULT_PATH).read())


Using exec() with file contents is a security risk and makes the code harder to test and debug. Consider refactoring process_result.py to expose functions that can be imported and called directly, rather than executing the entire script.

@copilot apply changes based on this feedback

Refactored process_result.py to expose a process_benchmark_result() function that can be imported and tested directly. Tests now call the function instead of using exec(), which is more secure and maintainable. All 15 tests still pass. Changes in commit f0353fc.

functionstackx · 2025-11-06T22:36:00Z

@copilot can u consolidate this pytest with .github/workflows/test-matrix-logic.yml

Copilot · 2025-11-06T22:39:23Z

@copilot can u consolidate this pytest with .github/workflows/test-matrix-logic.yml

Done! Consolidated both workflows into .github/workflows/test-utils.yml which now runs all utils tests:

73 matrix-logic tests
15 process_result tests

All 88 tests passing. Changes in commit 0a7e283.

Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>

…ixture Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>

Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>

Copilot AI assigned Copilot and functionstackx Nov 6, 2025

Copilot started work on behalf of functionstackx November 6, 2025 22:15 View session

functionstackx marked this pull request as ready for review November 6, 2025 22:24

functionstackx requested a review from a team as a code owner November 6, 2025 22:24

Copilot AI review requested due to automatic review settings November 6, 2025 22:24

Copilot AI reviewed Nov 6, 2025

View reviewed changes

Copilot AI changed the title ~~[WIP] Add unit test for utils/process_result.py and pytest CI~~ Add unit tests for utils/process_result.py with pytest CI Nov 6, 2025

Copilot AI requested a review from functionstackx November 6, 2025 22:27

Copilot finished work on behalf of functionstackx November 6, 2025 22:27

Copilot started work on behalf of functionstackx November 6, 2025 22:36 View session

Copilot AI changed the title ~~Add unit tests for utils/process_result.py with pytest CI~~ Add unit tests for utils/process_result.py with consolidated pytest CI Nov 6, 2025

Copilot finished work on behalf of functionstackx November 6, 2025 22:42

Copilot AI and others added 3 commits November 6, 2025 18:33

Add unit tests for utils/process_result.py and pytest CI workflow

b2f0b86

Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>

Improve test_process_result.py: use dynamic paths and remove unused f…

73b52a4

…ixture Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>

Consolidate pytest workflows into test-utils.yml

a204048

Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>

functionstackx force-pushed the copilot/write-unit-test-for-utils branch from 0a7e283 to a204048 Compare November 6, 2025 23:33

Update utils/test_process_result.py

88415bc

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot started work on behalf of functionstackx November 7, 2025 01:11 View session

Refactor process_result.py to use functions instead of exec()

f0353fc

Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>

Copilot finished work on behalf of functionstackx November 7, 2025 01:19

functionstackx closed this Dec 3, 2025

functionstackx deleted the copilot/write-unit-test-for-utils branch December 4, 2025 21:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add unit tests for utils/process_result.py with consolidated pytest CI#197

Add unit tests for utils/process_result.py with consolidated pytest CI#197
Copilot wants to merge 5 commits into
mainfrom
copilot/write-unit-test-for-utils

Copilot AI commented Nov 6, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 6, 2025

Uh oh!

Copilot AI Nov 6, 2025

Uh oh!

functionstackx Nov 7, 2025

Uh oh!

Copilot AI Nov 7, 2025

Uh oh!

Uh oh!

functionstackx commented Nov 6, 2025

Uh oh!

Copilot AI commented Nov 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Test Example

Comments on the Issue (you are @copilot in this section)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

functionstackx Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

functionstackx commented Nov 6, 2025

Uh oh!

Copilot AI commented Nov 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Nov 6, 2025 •

edited

Loading